Matrix basics

Dot product (projection product)

Properties

#TODO: add examples here

commutative
distribute over addition
associative over scalar multiplication

Cosine & dot product

A \cdot B = | A | | B | c o s θ

Projection

scalar projection (B projected on A) => a number (vector size) as result

\frac{A \cdot B}{| A |} = | B | c o s θ

vector projection (B projected on A) => a vector as result

\frac{A \cdot B}{| A |} \frac{A}{| A |} = \frac{A \cdot B}{A \cdot A} A = \frac{A \cdot B}{| A |^{2}} A

Changing basis

Basis is a set of n vectors which are linearly independent and form an n-dimensional space, i.e. if a, b, and c are linearly independent, for any $p_{a}$ and $p_{b}$ :

c \neq p_{a} a + p_{b} b

According to vector projection formula, project A on the new basis $(k_{1}, k_{2})$ :

A = \frac{A \cdot k_{1}}{| k_{1} |^{2}} k_{1} + \frac{A \cdot k_{2}}{| k_{2} |^{2}} k_{2}

thus, A in the new basis is:

A_{k} = [\frac{A \cdot k_{1}}{| k_{1} |^{2}}, \frac{A \cdot k_{2}}{| k_{2} |^{2}}]

Application of changing basis

PCA

Matrix multiplication

Matrices make transformations on vectors, potentially changing their magnitude and direction.

Key to solving simultaneous equation problems is appreciating how vectors are transformed by matrices, which is the heart of linear algebra.

Matrix Inverses

Gaussian elimination, also known as row reduction, is an algorithm in linear algebra for solving a system of linear equations.

For

A (\begin{array}{cc} a \\ b \\ c \end{array}) = S

The idea is using row operations on both A and S until A has a row echelon form:

(\begin{array}{cc} 1 & * & * \\ 0 & 1 & * \\ 0 & 0 & 1 \end{array})

then you can calculate $a, b, c$ .

Gaussian elimination can help in calculating inverse matrix $A^{- 1}$ of $A$ by tranforming:

(\begin{array}{cc} A | I \end{array})

(\begin{array}{cc} I | A^{- 1} \end{array})

Determinant

For

A = (\begin{array}{cc} a & b \\ c & d \end{array})

the determinant is

d e t (A) = | A | = a d - b c

when $d e t (A) = 0$ , $(\begin{array}{cc} a \\ c \end{array})$ and $(\begin{array}{cc} b \\ d \end{array})$ are linear dependent and unable to apply Gaussian elimination.